Skip to content

feat(planner): select summary-maintenance lifecycle plans - #318

Open
zzylol wants to merge 5 commits into
fix/summary-lifecycle-legalityfrom
feat/lifecycle-aware-selection
Open

feat(planner): select summary-maintenance lifecycle plans#318
zzylol wants to merge 5 commits into
fix/summary-lifecycle-legalityfrom
feat/lifecycle-aware-selection

Conversation

@zzylol

@zzylol zzylol commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Stacked on #317.

Why

Legal summary-maintenance alternatives must participate in candidate selection before a post-ASAP DAG is committed. The planner must account for both sides of demand:

  • query recurrence determines the expected number and timing of reads;
  • data arrival determines the expected number and timing of state updates.

Those quantities change whether rebuilding for every query, retaining shared state, continuously maintaining state, or evaluating the raw expression is cheapest.

Before this PR

The planner could enumerate lifecycle alternatives, but semantic summary candidates were still ranked without their complete build, update, read, retention, and retirement cost. The selected lifecycle and maintenance mode were not materialized as a physical guarantee.

After this PR

Every costable summary candidate receives a horizon-wide summary-maintenance cost before global selection. The selected post-ASAP DAG carries a SummaryMaintenanceLifecycleGuarantee, and materialization can still choose raw KeepPreAsap recomputation when it is no more expensive.

Selection algorithm

Inputs are the existing PlanSpace, the normalized QueryWorkload plus target-to-entry bindings, planning time, optional optimization horizon, runtime lifecycle capabilities, and the existing CostModel.

  1. Derive demand per target. PlanSpace propagates effective uses through the DAG and derives a RecurrenceProfile for each target. The lifecycle planner converts the bound workload entries into expected reads, one-time invocations, evaluation rate, fresh update rate, data arrival, and any valid prepared-state window.
  2. Enumerate each summary candidate. For every semantic Replacement::Summary candidate, collect each unique reachable SummaryAgg once by Rc identity.
  3. Enumerate lifecycle alternatives per state. Consider Ephemeral, Prepared, Shared, and ContinuouslyMaintained. Reject alternatives that violate workload requirements, runtime lifecycle support, summary update/delete capabilities, horizon requirements, or evidence freshness.
  4. Compute horizon-wide costs. The CostModel supplies primitive build, per-update maintenance, read, retention-rate, and retirement costs. The planner combines them with expected reads, update rate, and retention duration:
    • ephemeral rebuilds and retires state for every read;
    • prepared state is built before a known one-time execution window;
    • shared state pays one build and retention cost, then serves multiple reads;
    • continuously maintained state additionally pays for every expected update.
      With B = build, M = maintenance/update, Q = summary read, S = retention/second, X = retirement, R = expected reads, U = expected updates, and T = retained seconds, the implemented totals are:
    • ephemeral: R * (B + Q + X);
    • prepared: B + one_time_reads * Q + U * M + T * S + X;
    • shared or continuously maintained: B + R * Q + U * M + T * S + X.
      For data at rest, U * M is zero. Missing inputs remain unknown; they are never interpreted as zero.
  5. Make nested summaries executable together. Summary states connected without an intervening readout must use a compatible EvaluationSchedule. For each connected component, compare the total cost under each schedule that every member supports, choose the cheapest common schedule, then choose each member's cheapest lifecycle under that schedule.
  6. Rank semantic summary candidates. Sum the selected costs of unique deployments and attach the known total as a CandidateCostOverrides entry. PlanSpace::global_selection_with_candidate_costs uses the cheapest lifecycle-aware summary candidate for that memo group while retaining its existing topological effective-use propagation.
  7. Materialize the physical guarantee. Materialize the globally selected DAG and attach lifecycle, SummaryMaintenanceMode, evaluation schedule, and output representation to each deployment.
  8. Compare the complete summary plan with raw recomputation. raw_query_recompute_cost * expected_reads is compared with the selected summary deployment total. A known raw cost wins ties and replaces the target with KeepPreAsap.

How this composes with the existing cost planner

This is an additional physical-cost stage, not a replacement cost system:

  • Replacement strategies and accuracy/error legality first determine which logical rewrites and summary algorithms are valid candidates.
  • Existing sketch sizing and semantic candidate construction still use the same CostModel.
  • Existing recurrence-aware CSE planning still provides recurrence profiles and propagates effective consumer counts. It continues to decide share versus recompute for groups that do not receive a lifecycle-cost override. For details, see PR #295, the cost semantics and RecurrenceProfile documentation in recurrence.rs, and the selection code in PlanSpace::global_selection_with_recurrence and decide_group_with_recurrence.
  • Lifecycle planning reuses those recurrence profiles and adds complete physical summary-state costs through CandidateCostOverrides.
  • If a memo group has no candidate with a known lifecycle total, that group falls back to the existing estimate_cost/CSE ranking path. Within a lifecycle-ranked group, uncosted candidates cannot beat known totals; unknown evidence never becomes an artificial zero.
  • The final raw comparison is a whole-target fallback after materialization, rather than another summary lifecycle.

Therefore the flow is:

candidate legality and sizing -> recurrence/effective-use analysis -> lifecycle-aware candidate totals -> global PlanSpace selection -> physical lifecycle guarantee -> raw recomputation fallback.

Verification

Stack

Base: #317. Next: #319.

@zzylol
zzylol force-pushed the fix/summary-lifecycle-legality branch from 9806c0a to 495b664 Compare August 30, 2026 18:21
@zzylol
zzylol force-pushed the feat/lifecycle-aware-selection branch from 51e1904 to e716fed Compare August 30, 2026 18:21
@zzylol
zzylol force-pushed the fix/summary-lifecycle-legality branch from 495b664 to 4015f26 Compare August 30, 2026 18:30
@zzylol
zzylol force-pushed the feat/lifecycle-aware-selection branch from e716fed to a18c539 Compare August 30, 2026 18:30
@zzylol
zzylol force-pushed the fix/summary-lifecycle-legality branch from 4015f26 to 944df5d Compare August 30, 2026 18:35
@zzylol
zzylol force-pushed the feat/lifecycle-aware-selection branch from a18c539 to 06a2099 Compare August 30, 2026 18:35
@zzylol
zzylol force-pushed the fix/summary-lifecycle-legality branch from 944df5d to 621fb5e Compare August 30, 2026 18:46
@zzylol
zzylol force-pushed the feat/lifecycle-aware-selection branch from 06a2099 to 3293d37 Compare August 30, 2026 18:46
@zzylol
zzylol force-pushed the fix/summary-lifecycle-legality branch from 621fb5e to 5adb5f1 Compare August 30, 2026 19:03
@zzylol
zzylol force-pushed the feat/lifecycle-aware-selection branch from 3293d37 to c8f7c0c Compare August 30, 2026 19:03
@zzylol
zzylol force-pushed the fix/summary-lifecycle-legality branch from 5adb5f1 to 158002a Compare August 30, 2026 21:22
@zzylol
zzylol force-pushed the feat/lifecycle-aware-selection branch from c8f7c0c to 0613a90 Compare August 30, 2026 21:22
@zzylol
zzylol force-pushed the fix/summary-lifecycle-legality branch from 158002a to 3df3599 Compare August 30, 2026 21:27
@zzylol
zzylol force-pushed the feat/lifecycle-aware-selection branch from 0613a90 to c517a1a Compare August 30, 2026 21:27
@zzylol
zzylol force-pushed the fix/summary-lifecycle-legality branch from 3df3599 to c9334fc Compare August 31, 2026 12:04
@zzylol
zzylol force-pushed the feat/lifecycle-aware-selection branch 2 times, most recently from bc90946 to 2cce2eb Compare August 31, 2026 12:17
@zzylol
zzylol force-pushed the fix/summary-lifecycle-legality branch from c9334fc to 5e564bd Compare August 31, 2026 12:17
@zzylol zzylol changed the title feat(planner): select lifecycle-aware summary plans feat(planner): select summary-maintenance lifecycle plans Aug 31, 2026
@zzylol
zzylol force-pushed the fix/summary-lifecycle-legality branch from 5e564bd to 8db7940 Compare August 31, 2026 12:35
@zzylol
zzylol force-pushed the feat/lifecycle-aware-selection branch 2 times, most recently from b162a36 to b031b7f Compare August 31, 2026 12:45
@zzylol
zzylol force-pushed the fix/summary-lifecycle-legality branch from 8db7940 to 27296e7 Compare August 31, 2026 12:45
@zzylol
zzylol force-pushed the feat/lifecycle-aware-selection branch from b031b7f to d77479b Compare August 31, 2026 21:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant